Fast Exact Bayesian Inference for Sparse Signals in the Normal Sequence Model

We consider exact algorithms for Bayesian inference with model selection priors (including spike-and-slab priors) in the sparse normal sequence model. Because best existing algorithm becomes numerically unstable sample sizes over n=500, there has been much attention alternative approaches like approximate (Gibbs sampling, variational Bayes, etc.), shrinkage (e.g. Horseshoe prior and Spike-and-Slab LASSO) or empirical methods. However, by introducing algorithmic ideas from online sequential prediction, we show that calculations are feasible larger sizes: general reach n=25000, certain can easily n=100000. further prove a de Finetti-like result finite characterizes exactly which be expressed as priors. The computational speed numerical accuracy of proposed methods demonstrated experiments on simulated data, differential gene expression data set, to compare effect multiple hyper-parameter settings beta-binomial prior. In our experimental evaluation compute guaranteed bounds all new algorithms, shows reliable whereas an based long division is not.